IMAGE PROCESSING APPARATUS AND METHOD, AND 
COMPUTER READABLE STORAGE MEDIUM 



BACKGROUND OF THE INVENTION 

5 

Field of the Invention 

The present invention relates to an image processing apparatus and method for 
correcting a variation of an image signal and performing the coding of the image signal, 
and a computer readable storage medium storing a program for implementing the 
10 method. 



Description of the Related Art 

In a conventional method for coding image data using a stable synchronizing 
signal, an input image signal is input into a time base corrector so as to correct variations 
15 in the signal, and then image data in which such a variation has been corrected is coded 
in an image coding apparatus. The above variations include (i) dispersion of the 
transmission speed of the input image signal, (ii) disturbances of the synchronizing 
signal due to switching of scenes in the input image signal. 

Fig. 4 is a block diagram showing the structure of a conventional time base 
20 corrector and image coding apparatus. 

In Fig. 4, an analog image signal Bl such as an image signal obtained using a 
video camera, a recorded signal (to be reproduced) of a VTR, or a TV broadcast signal is 
input into a video decoder Al, where the analog signal is AID converted and a digital 
image signal (called "image data" hereinbelow) B2 is obtained. In addition, an 
25 image-input clock and synchronizing signal B3 in synchronism with the image data B2 



is also output from the video decoder Al. Therefore, these image data B2 and 
image-input clock and synchronizing signal B3 include a variation of the analog image 
signal Bl. 

Next, these image data B2 and image-input clock and synchronizing signal B3, 
5 including such a variation, are input into a time base corrector A8. In the time base 
corrector A8, the image writing section AlO writes image data B2 via image memory 
interface Al 1 into image memory A9 as image data B7. In this process, an image 
reading and writing control section A12 controls the data writing operation based on the 
clock signal (including a variation) output from the image writing section AlO. 

10 The image memory A9 has, for example, a storage capacity of 2 jBrames, and 

when a frame of image data B7 is written, the image reading section Al 3 performs the 
data reading operation and the next frame of image data is written. As the writing 
operation is accompanied by the reading operation, image data B8 is obtained. This 
image data B8 is read via image memory interface All by an image reading section A13. 

15 hi this process, the image reading and writing control section A12 controls the data 
reading operation based on a stabilized clock signal output from a clock generator A19. 
Therefore, stable image data B9 in which variations of the input image signal are 
corrected, and a stable image-input clock and synchronous signal BIO are output from 
the time base corrector A8. The output signals are then input into an image coding 

20 apparatus A 14. 

Jn the image coding apparatus A14, an image writing section A16 writes the 
above image data B9 via an image memory interface A17 into an image memory Al 5 as 
image data B12. This writing operation is performed using the above clock signal B 11 . 
A coding section A18 then reads image data B13 via image memory interface A17 from 

25 the image memory Al 5 by using the clock signal B 1 1 , and codes the read data. Here, 



3 

the reading operation from image memory A15 is executed for each coding unit, for 
example, for each Macro Block including 16 X 16 pixels, and the coding section A18 
executes the coding operation in Macro-Block units. Accordingly, coded and 
compressed image data B6 can be obtained. 
5 Fig. 5 is a block diagram showing the detailed structure of an example of the 

image writing and reading control section Al 2 in the time base corrector A8. In the 
figure, the image writing and reading control section A12 includes a writing line number 
threshold register A121, a reading hne number threshold register A122, and a 
comparator A 123. 

10 Fig. 6A is a flowchart of the operation of writing data into image memory A9, 

while Fig. 6B is a flowchart of the operation of reading data from image memory A9. 

Both operations are performed by the image writing and reading control section A 12. 

For writing and reading control of the image memory A9, it is necessary to (i) 

prevent the writing operation from going ahead of the reading operation and thus 
1 5 deleting necessary data, and (ii) prevent the reading operation from going ahead of the 

writing operation and thus again reading out a previous frame during the reading of the 

current frame. 

With reference to Figs. 5 and 6A, it is assumed that the writing of a frame of 
image data is executed, as shown in step S 1 1 . Simultaneously, the reading of image 
20 data is performed (see step S21). When the writing of the frame is completed (see step 
S12), the comparator A123 compares the reading line number LI received from the 
image reading section A13 with the threshold L2 received from the reading line number 
threshold register A122 (see step S13). The reading line number LI indicates the 
number of lines in the current frame which have ahready been read. If LI > L2, the 
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operation is returned to step S 1 1 and the writing operation is continued, while if LI ^ 
L2, it is determined that the writing operation may go ahead of the reading operation, 
and in step SI 4, a frame is skipped in the writing operation. 

Next, with reference to Figs. 5 and 6B, it is assumed that the reading of a frame 

5 of image data is executed, as shown in step S21 . Simultaneously, the writing of image 
data is performed (see step 811). When the reading of the frame is completed (see step 
S22), the comparator A 123 compares the writing line number L3 received from the 
image writing section AlO with the threshold L4 received from the writing line number 
threshold register AI21 (see step S23). The writing line number L4 indicates the 

10 number of lines in the current frame which have already been written. IfL3 > L4, 
the operation is returned to step S21 and the reading operation is continued, while if L3 
^ L4, it is determined that the reading operation may go ahead of the writing operation, 
and in step 824, the current frame is again read in the reading operation. 

Japanese Unexamined Patent Application, First Publication, No. Hei 8-223567, 

1 5 discloses an example of the above-explained conventional technique. In the coding 
apparatus disclosed in this document, a frame synchronizer is connected to a 
high-efficiency coding section, and each of the frame synchronizer and the 
high-efficiency coding has an image memory. 

As explained above, in the conventional technique, a variation of image data is 

20 corrected using a time base corrector, and then the data is coded in an image coding 
apparatus. Therefore, a set of an unage memory, an image writing section, and an 
image memory interface is necessary for each of the time base corrector and the image 
coding apparatus. Accordingly, the memory must have a large capacity, and the circuit 
arrangement must be complicated. 



5 

In addition, to provide two image memories causes an increase of the execution 
number of the data writing and reading operation, thereby increasing the processing time 
from the image input to the start of the coding. 



5 SUMMARY OF THE INVENTION 

In consideration of the above circumstances, an objective of the present 
invention is to provide an image processing apparatus in which a time base corrector and 
an image coding apparatus are integrated and the capacity of the memory is reduced, 
thereby realizing a simple circuit arrangement. 
1 0 Therefore, the present invention provides an image processing apparatus 

comprising: 

an image memory for storing image data; and 

a control section for performing control of writing each line of input image data 
to the image memory in turn while reading image data from the image memory in 
1 5 predetermined coding units, in a manner such that: 

when a picture of the image data has been written, if the number of 
data units, each corresponding to the coding vmit, is equal to or below a first threshold, 
then a picture is skipped in the image data writing operation; and 

when a picture of the unage data has been read, if the number of 
20 written lines is equal to or below a second threshold, then the same picture is again read 
in the image data reading operation. 

According to the above control in the writing and reading of image data into 
and from the image memory (provided between an image writing section and a coding 
section), an image coding apparatus and a time base corrector can be integrated, so that 
25 the coding operation can be performed using a single memory included in the integrated 
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image coding apparatus. More specifically, deletion or insufficiency of necessary data 
in a memory in the coding operation can be prevented (such deletion or insufficiency 
may occur in a conventional image coding apparatus without a time base corrector). In 
addition, in comparison with the conventional structure in which the image coding 
5 apparatus and the time base corrector respectively use different memories, a single 
memory can be used in common, thereby reducing the memory capacity. 

Furthermore, a simple circuit structure can be realized, and the processing time 
can be reduced. 

Preferably, the image data writing operation is performed using a clock signal 
10 in synchronism with the input image data; and the image data reading operation is 
performed using a stabilized clock signal different from the clock signal used in the 
image data writing operation. 

The image processing apparatus may further comprise a coding section for 
coding the read image data in the predetermined coding units. 
1 5 The predetermined coding unit may be a Macro Block defined in MPEG. 

Typically, a picture corresponds to a frame or a field of image data. 

The present invention also provides an image processing method comprising 
the step of: 

performing control of writing each line of input image data to an image 
20 memory in turn while reading image data fi:om the image memory in predetermined 
coding units, in a manner such that: 

when a picture of the image data has been written, if the number of 
data units, each corresponding to the coding imit, is equal to or below a first threshold, 
then a picture is skipped in the image data writing operation; and 
25 when a picture of the image data has been read, if the number of 
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written lines is equal to or below a second threshold, then the same picture is again read 
in the image data reading operation. 

The present invention also provides a computer readable storage medium 
storing a program for making a computer execute an operation including the step of: 
5 performing control of writing each line of input image data to an image 

memory in turn while reading image data from the image memory in predetermined 
coding units, in a manner such that: 

when a picture of the image data has been written, if the number of 
data units, each corresponding to the coding unit, is equal to or below a first threshold, 
1 0 then a picture is skipped in the image data writing operation; and 

when a picture of the image data has been read, if the number of 
written lines is equal to or below a second threshold, then the same picture is again read 
in the image data reading operation. 

1 5 BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing the structure of the image processing 
apparatas as an embodiment according to the present invention. 

Fig. 2 is a block diagram showing the detailed structure of the image writing 
and reading control section in the image processing apparatus of Fig. 1. 
20 Fig. 3 A is a flowchart of the operation of writing data into the image memory in 

the image processing apparatus, and Fig. 3B is a flowchart of the operation of reading 
data from the image memory. 

Fig. 4 is a block diagram showing the structure of a conventional time base 
corrector and image coding apparatus. 
25 Fig. 5 is a block diagram showing the detailed structure of the image writing 
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and reading control section in the time base corrector of Fig. 4. 

Fig. 6 A is a flowchart of the operation of writing data into the image memory in 
the conventional time base corrector, and Fig. 6B is a flowchart of the operation of 
reading data from the image memory. 

5 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Hereinafter, an embodiment according to the present invention will be 
explained in detail with reference to the drawings. 

Fig. 1 is a block diagram showing the structure of the embodiment of the 
1 0 present invention, that is, of an image processing apparatus A2 in which a time base 

corrector and an image coding apparatus are integrated, and which functions as an image 
coding apparatus having the function of a time base corrector. 

In Fig. 1, an analog image signal Bl such as an image signal obtained using a 
video camera, a reproduced signal of a VTR, or a TV broadcast signal is input into a 
1 5 video decoder Al, where the analog signal is AID converted and image data B2 is 
obtained. In addition, an image-input clock and synchronizing signal B3 in 
synchronism with the image data B2 is also output from the video decoder Al . As 
explained above, these image data B2 and image-input clock and synchronizing signal 
B3 include a variation of the analog image signal Bl . 
20 Next, these image data B2 and image-input clock and synchronizing signal B3 

are input into an image processing apparatus A2. An image writing section A3 writes 
image data B2 via image memory interface A4 into image memory A7 as image data B4. 
In this process, the image writing section A3 and an image reading and writing control 
section A5 control the data writing operation based on a clock signal in synchronism 
25 with the image-input clock and synchronizing signal B3. 



When a frame of image data B4 is first written into the image memory A7, the 
data reading operation is performed by the image reading and writing control section A5, 
and then the data reading is continued while the data writing operation is also performed. 
The read image data B5 is input via the image memory interface A4 into a coding 
section A6. In this case, the data in image memory A7 is read in coding units (i.e., 
each data unit corresponding to the coding unit is read in tum). For example, in the 
case of MPEG, data corresponding to each Macro Block including 16X16 pixels is read, 
and the coding section A6 codes data in Macro Block vmits according to a specific 
coding method such as MPEG2. Accordingly, coded and compressed image data B6 
can be obtained. 

Li addition, the image reading and writing control section A5 performs reading 
control and coding operation by using a stabilized clock signal Bll supplied by a clock 
generator All which generates reference clock data. 

Fig. 2 is a block diagram showing the detailed structure of the image writing 
and reading control section A5 in the image processing apparatus A2. In the figure, the 
image writing and reading control section A5 includes a writing line number threshold 
register A5 1 , a reading Macro Block number threshold register A52, and a comparator 
A53. 

Fig. 3 A is a flowchart of the operation of writing data into image memory A7, 
while Fig. 3B is a flowchart of the operation of reading data from image memory AT. 
Both operations are performed by the image writing and reading control section A5, and 
are provided for keeping a balance of the writing and reading operations. 

With reference to Figs. 2 and 3 A, it is assumed that the writing of a frame of 
image data is executed, as shown in step SI . Simultaneously, the reading of image data 
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is performed (see step S21). When the writmg of the frame is completed (see step S2), 
the comparator A53 compares a reading Macro Block number L5 received from the 
coding section A6 with threshold L6 received from the reading Macro Block number 
threshold register A52 (see step S3). The reading Macro Block number L5 indicates 

5 the number of Macro Blocks in the current frame which have already been read. If L5 
> L6, the operation is returned to step SI and the writing operation is continued, while 
if L5 ^ L6, it is determined that the writing operation may go ahead of the reading 
operation, and in step S4, a frame is skipped in the writing operation. 

Next, with reference to Figs. 2 and 3B, it is assumed that the reading of a frame 

10 of image data is executed, as shown in step S21, Simultaneously, the writing of image 
data is performed (see step S 1). When the reading of the frame is completed (see step 
S22), the comparator A53 compares the writmg line number L3 received from the image 
vmting section A3 with the threshold lA received from the writing line number 
threshold register A5 1 (see step S23). The writing line number L3 indicates the 

1 5 number of lines in the current frame which have already been written. If L3 > L4, 
the operation is returned to step S21 and the reading operation is continued, while if L3 
^ L4, it is determined that the reading operation may go ahead of the writing operation, 
and in step S24, the current frame is again read in the reading operation. 

In the present embodiment, a pictiare corresponds to a frame. However, a 

20 picture may correspond to a field. 

As explained above, according to the above embodiment, each line of image 
data is written into the image memory A7 by using a clock signal in synchronism with 
an input image signal, and simultaneously, data is written in Macro Block units by using 
a stabilized clock signal. Here, the writing and reading operation is controlled while 
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the number of written lines is compared with a predetermined threshold, and the number 
of read Macro Blocks is compared with a predetermined threshold. Therefore, the 
writing operation does not go ahead of the reading operation, and the reading operation 
does also not go ahead of the writing operation, and it is possible to prevent the current 
5 frame from being erroneously switched to another frame while the current frame is being 
processed. 

In the conventional technique, a plurality of image memory interfaces, image 
writing section, and the like are necessary. However, in the present embodiment, a 
common image memory A7 is used, so that such duplication of circuit elements is 
1 0 unnecessary, and it is possible to realize an image coding apparatus having the function 
of a time base corrector, which has a simple structure and whose processing time is 
short. 

More specifically, it is assumed that in the conventional apparatus as shown in 
Fig. 4, a memory capacity of 2 frames (or 2 fields, a corresponding variation will also be 
1 5 applied below) is assigned to each of the image memory A9 and image memory Al 5, 
that is, a capacity of 4 frames is provided. In this case, in the present embodiment, 
image memory A7 needs a memory capacity of 3 frames, and accordingly, only a single 
image memory interface A4, a single image writing section A3, and the like are 
necessary. 

20 If an image processing apparatus (as shown in Fig. 1) is realized in a computer 

system having a CPU or MPU, a memory, and the like, the memory functions as a 
computer readable storage mediirai according to the present invention. In this storage 
medium, program codes for executing an operation, which includes the operation shown 
in Figs. 3A and 3B, are stored. In addition, various types of storage media such as a 

25 disk type medium, a semiconductor memory, and the like, may be used as the above 



storage medium. 



